Using Kernel PCA for Initialisation of Nonlinear Factor Analysis

The nonlinear factor analysis (NFA) method by Lappalainen and Honkela (2000) [2] is initialised with linear principal component analysis (PCA). Because of the multilayer perceptron (MLP) network used to model the nonlinearity, the method is susceptible to local minima and therefore sensitive to the initialisation used. As the method is used for nonlinear separation, the linear initialisation may in some cases lead it astray. In this report we study using kernel PCA (KPCA) to initialise NFA. KPCA is a rather straightforward generalisation of linear PCA and it is much faster to compute than NFA. The experiments show that it may produce significantly better initialisations than linear PCA, although finding a suitable kernel and parameters may be difficult.